Skip to content

[SPARK-51463] Add Spark Connect-generated Swift source code #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

[SPARK-51463] Add Spark Connect-generated Swift source code #5

wants to merge 2 commits into from

Conversation

dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Mar 11, 2025

What changes were proposed in this pull request?

Like other Spark Connect client library, this PR aims to add Spark Connect-generated Swift source code.

These will be replaced by Swift Plugin later.

Why are the changes needed?

These files are manually generated from v4.0.0-rc2 tag by the following standard procedures. This will be updated when Apache Spark 4.0.0 is released officially.

$ git clone -b v4.0.0-rc2 https://github.com/apache/spark.git
$ cd spark/sql/connect/common/src/main/protobuf/
$ protoc --swift_out=. spark/connect/*.proto
$ protoc --grpc-swift_out=. spark/connect/*.proto

// Remove empty GRPC files
$ git grep 'This file contained no services'
spark/connect/catalog.grpc.swift:// This file contained no services.
spark/connect/commands.grpc.swift:// This file contained no services.
spark/connect/common.grpc.swift:// This file contained no services.
spark/connect/example_plugins.grpc.swift:// This file contained no services.
spark/connect/expressions.grpc.swift:// This file contained no services.
spark/connect/ml.grpc.swift:// This file contained no services.
spark/connect/ml_common.grpc.swift:// This file contained no services.
spark/connect/relations.grpc.swift:// This file contained no services.
spark/connect/types.grpc.swift:// This file contained no services.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass the CIs.

Was this patch authored or co-authored using generative AI tooling?

No.

@@ -15,5 +15,6 @@ header:
- '.asf.yaml'
- '.nojekyll'
- 'Package.swift'
- '**/*pb.swift'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These generated files contains ASF license headers but starts with additional text patterns.

// DO NOT EDIT.
// swift-format-ignore-file
// swiftlint:disable all
...

@dongjoon-hyun
Copy link
Member Author

Could you review this Spark Connect-generated Swift stuff PR when you have some time, @LuciferYang ?

Copy link

@LuciferYang LuciferYang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dongjoon-hyun
Copy link
Member Author

Thank you so much, @LuciferYang !

@dongjoon-hyun
Copy link
Member Author

Merged to main.

@dongjoon-hyun dongjoon-hyun deleted the SPARK-51463 branch March 11, 2025 05:47
dongjoon-hyun added a commit that referenced this pull request May 12, 2025
…th 4.0.0-rc5

### What changes were proposed in this pull request?

This PR aims to update `Spark Connect`-generated `Swift` source code with `Apache Spark 4.0.0-rc5`.

### Why are the changes needed?

SPARK-51463 added this code with `Apache Spark 4.0.0-rc2`.
- #5

There were some changes so far like the following.
- apache/spark#50441

The following is the procedure.
```
$ git clone -b v4.0.0-rc5 https://github.com/apache/spark.git
$ cd spark/sql/connect/common/src/main/protobuf/
$ protoc --swift_out=. spark/connect/*.proto
$ protoc --grpc-swift_out=. spark/connect/*.proto

// Remove empty GRPC files
$ git grep 'This file contained no services'
spark/connect/catalog.grpc.swift:// This file contained no services.
spark/connect/commands.grpc.swift:// This file contained no services.
spark/connect/common.grpc.swift:// This file contained no services.
spark/connect/example_plugins.grpc.swift:// This file contained no services.
spark/connect/expressions.grpc.swift:// This file contained no services.
spark/connect/ml.grpc.swift:// This file contained no services.
spark/connect/ml_common.grpc.swift:// This file contained no services.
spark/connect/relations.grpc.swift:// This file contained no services.
spark/connect/types.grpc.swift:// This file contained no services.
```

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Pass the CIs.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #134 from dongjoon-hyun/SPARK-52083.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants